Apparatus, systems and methods for layout of scene graphs using node bounding areas

ABSTRACT

A value is assigned to a layout bound of a first node in a scene graph. The layout bound constitutes a bounding volume for the object corresponding to the node and may be the display properties of the object and a first set of display modifiers for the node but not a second set. A display layout is calculated for a second node in the scene graph based on the value of the layout bound. Then, nodes of the scene graph are rendered to generate a display on a display device according to the calculated display layout. The value of the layout bound may be assignable, creating greater flexibility in controlling layout. Additionally, the value assigned to the layout bound may be changed. In this way, layout of nodes with respect to each other is flexible and visual effects and animations can either be factored into that layout or not.

FIELD OF THE INVENTION

Aspects of this invention relate generally to computer systems, and morespecifically to displays.

BACKGROUND

A scene graph is a data structure commonly used by vector-based displayapplications. The scene graph is typically a collection of nodes in agraph or tree structure. A node typically include display properties foran object associated with the node and one or more sets of displaymodifiers such that rendering the node includes applying the displaymodifiers to the display properties to display the object. A node mayhave children but often only a single parent. A parent node may renderits children nodes and may propagate properties or modifiers to itschildren. Additionally, nodes in a scene graph may be grouped such thatan operation applied to a group automatically propagates to its members.In some scene graphs, a node can have a relation to other nodes, perhapseven itself.

When creating rich displays using a scene graph, such as visualinterfaces, it is often desirable to support the ability to lay outnodes with respect to each other, as well as define visual effects andanimations that are either factored into that layout or are not.Typically, scene graph systems struggle to support both of theseconcepts. This is further complicated when the displays are dynamicrather than merely static.

SUMMARY

Aspects of the present disclosure provide apparatus, systems and methodsfor layout of scene graphs for generating displays using node boundingareas. A value is assigned to a layout bound of a first node in a scenegraph. The layout bound of the first node may constitute a boundingvolume for the object corresponding to the node. A display layout iscalculated for a second node in the scene graph based on the value ofthe layout bound. Then, at least the first node and second node arerendered to generate a display on a display device according to thecalculated display layout. Thus, lay out of nodes is based on layoutbounds of nodes rather than physical display properties of the objects.

The value of the layout bound may be assignable, creating greaterflexibility in controlling layout. In one or more embodiments, the valueassigned to the layout bound for the first node may be the displayproperties of the object corresponding to the node and a first set ofdisplay modifiers for the node, but not a second set of displaymodifiers for the node. In this way, layout of nodes with respect toeach other is flexible and visual effects and animations can either befactored into that layout or not.

Additionally, the value assigned to the layout bound may be changed.This provides much greater flexibility in layout than scene graphssystems that lay out nodes with respect to each other based on thephysical display properties of the objects corresponding to the nodes.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not necessarily restrictive of the present disclosure. Theaccompanying drawings, which are incorporated in and constitute a partof the specification, illustrate subject matter of the disclosure.Together, the descriptions and the drawings serve to explain theprinciples of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure may be better understood, and its numerousfeatures made apparent to those skilled in the art by referencing theaccompanying drawings.

FIG. 1 is a block diagram illustrating a system 100 for layout of scenegraphs for generating displays using node bounding areas, in accordancewith one or more embodiments of the present disclosure;

FIG. 2 is a method diagram illustrating a method 200 for layout of scenegraphs for generating displays using node bounding areas, which may beperformed by the system 100 of FIG. 1, in accordance with one or moreembodiments of the present disclosure;

FIG. 3A is a block diagram illustrating the performance of an exampleimplementation of the method of FIG. 2;

FIG. 3B is a block diagram illustrating the performance of an exampleimplementation of the method of FIG. 2; and

FIG. 3C is a block diagram illustrating the performance of an exampleimplementation of the method of FIG. 2.

DETAILED DESCRIPTION

Aspects of the present disclosure provide apparatus, systems and methodsfor layout of scene graphs for generating displays using node boundingareas. A value is assigned to a layout bound of a first node in a scenegraph. A display layout is calculated for a second node in the scenegraph based on the value of the layout bound. Then, at least the firstnode and second node are rendered to generate a display on a displaydevice according to the calculated display layout. Thus, lay out ofnodes is based on layout bounds of nodes, which may constitute abounding volume for the objects corresponding to the nodes, rather thanphysical display properties of the objects. The value of these layoutbounds are assignable, creating greater flexibility in controllinglayout. The value assigned to the layout bound for a node may be thedisplay properties of the object corresponding to the node and a firstset of display modifiers for the node, but, in one example, not a secondset of display modifiers for the node. In this way, layout of nodes withrespect to each other is flexible and visual effects and animations caneither be factored into that layout or not. Additionally, the valueassigned to the layout bound may be changed, providing much greaterflexibility in layout than scene graphs systems that lay out nodes withrespect to each other based on the physical display properties of theobjects corresponding to the nodes.

Typically, scene graphs systems lay out nodes with respect to each otherbased on the physical display properties of the objects corresponding tothe nodes. These conventional scene graph systems are typically unableto support both the ability to lay out nodes with respect to each otherand define visual effects and animations that are either factored intothat layout or are not.

FIG. 1 illustrates a system 100 for layout of scene graphs forgenerating displays using node bounding areas, in accordance with anembodiment of the present disclosure. The system 100 involves acomputing device 101 coupled to a display device 102. The computingdevice 101 includes a processing unit 103, which is operable to executeinstructions implementing a display generator 105. The processing unit103 may constitute a graphics processing unit, a visual processing unit,or a graphics accelerator. The computing device 101 also includes atangible machine-readable media 104, communicably coupled to theprocessing unit 103, operable to store a scene graph 106. The tangiblemachine-readable media 104 may constitute any tangible media capable ofstoring electronic information, such as a memory or a hard disk drive.

The computing device 101 may include any kind of computing device suchas a personal computer, a server computer, a laptop computer, a personaldigital assistant, or a cellular telephone. The computing device 101 mayalso include (not shown) one or more input and/or output devices(including, but not limited to keyboards, mice, printers, scanners, andso forth), one or more busses (including, but not limited to, universalserial bus, small computer system interface, and so forth), and/or oneor more communication components (including, but not limited to, modems,Ethernet adapters, wireless Ethernet adapters, and so forth). Thedisplay device 102 may include any kind of display device such as acathode ray-tube display, a liquid crystal display, or a plasma screendisplay.

It is understood that the system 101 is merely an example, and that thesystem 101 may include more than one computing device 101, displaydevice 102, processing unit 103, and/or tangible machine-readable media104 without departing from the scope of the present disclosure.Additionally, though the computing device 101 and the display device 102are shown as separate devices, it is understood that the computingdevice 101 and the display device 102 may be integrated into a singledevice without departing from the scope of the present disclosure.Moreover, although the processing unit 103 and the machine-readablemedia 104 are illustrated as separate components of the computing device101, they may be incorporated into a single component of the computingdevice 101, such as a graphics card, a video card, or a motherboard withon-board video or on-board graphics.

The display generator 105, operating on the processing unit 103, isconfigured to generate a display on the display device 102 by renderingthe scene graph 106 stored in the tangible machine-readable media 104.The display generator 105 may be configured to render the scene graph106 by rendering one or more nodes included in the scene graph 106. Thedisplay may constitute a user interface, such as a graphical userinterface. The display may also constitute a vector graphics display.

FIG. 2 illustrates a method 200 for layout of scene graphs forgenerating displays using node bounding areas, which may be performed bythe system 100, in accordance with one or more embodiments of thepresent disclosure. The method 200 may comprise a computer-implementedmethod, with the method performed by a processing unit, such as theprocessing unit 103, executing one or more sets of instructions includedin a computer program product stored in a machine-readable media, suchas the tangible machine-readable media 104.

In a first operation 210, the display generator 105 may assign a valueto a layout bound to a first node in a plurality of nodes in the scenegraph 106. The first node may include display properties of an objectcorresponding to the first node and one or more sets of displaymodifiers. The object may include any kind of object corresponding tothe first node, such as a window, a toolbar, a rectangle, a circle, atriangle, and/or a polygon. The display properties of the object mayinclude specifications as to how the object is to be displayed, such asa specified area (i.e., height and width of the object), position,color, opacity, or fill. The one or more sets of display modifiers mayinclude mathematical transformations configured to modify the displayproperties. For example, the one or more sets of display modifiers mayinclude mathematical transformations to scale, skew, translate, reflect,or shear the specified area of the object. By way of another example,the one or more sets of display modifiers may include mathematicaltransformations to apply a visual effect or animation to the displayproperties of the object. The visual effects or animations may includevisual effects or animations to apply a pulsating glow to the object,apply a zoom-fade to the object, apply a fade in to the object, apply afade out to the object, cause the object to wiggle, apply a drop shadowto the object, applying a zoom-out to the object, applying a slide tothe object, and etc. Rendering the first node may include applying theone or more sets of display modifiers to the display properties.

The layout bound may constitute a bounding volume for the objectcorresponding to the first node. A bounding volume for an object is aclosed volume that completely contains the object. The layout bound mayconstitute any kind of bounding volume, such as a bounding sphere, abounding cylinder, a bounding capsule, a bounding box, a minimumbounding rectangle, a discrete oriented polytope, and/or a convex hull.The layout bound may encompass the specified area of the object.

The one or more sets of display modifiers may include a first set ofdisplay modifiers and a second set of display modifiers. The valueassigned to the layout bound may include the display properties of theobject and the first set of display modifiers but not the second set ofdisplay modifiers. Hence, if a display modifier in the first set ofdisplay modifiers is altered, the display layout may change because thelayout bound has changed. Further, if a display modifier in the secondset of display modifiers is altered, the display layout may not changebecause the layout bound has not changed. Alternatively, the valueassigned to the layout bound may include a value or expression otherthan the display properties of the object and the first set of displaymodifiers but not the second set of display modifiers.

In a second operation 220, the display generator 105 may calculate adisplay layout for at least a second node in the plurality of nodesrelative to the first node based on the value of the layout bound. Thescene graph may include specifications as to render the second noderelative to the first node such as a distance and/or position from thefirst node, at least a distance and/or position from the first node, orwithin a distance and/or position from the first node. For example, thescene graph 106 may include a specification that the second node berendered on the display ten pixels to the right and ten pixels above thefirst node. In this case, the display generator 105 may calculate thedisplay layout to render the second node ten pixels to the right and tenpixels above the layout bound of the first node. The display generator105 may include a layout manager component 107 and may calculate thedisplay layout utilizing the layout manager 107. The layout manager 107may read the specifications included in the scene graph as to render thesecond node relative to the first node and may calculate a displaylayout for the second node that complies with the specifications.

In a third operation 230, the display generator 105 may render at leastthe first node and the second node to generate a display on the displaydevice 102 according to the calculated display layout. The display mayconstitute a user interface, such as a graphical user interface. Thedisplay may also constitute a vector graphics display.

In some embodiments, the method 200 may include operations 240-260. Inthe fourth operation 240, the display generator 105 may assign a newvalue to the layout bound. If the value assigned to the layout bound wasthe display properties of the object and the first set of displaymodifiers but not the second set of display modifiers, the new valueassigned to the layout bound may include a value or expression otherthan the display properties of the object and the first set of displaymodifiers but not the second set of display modifiers. For example thenew value assigned to the layout bound may include the displayproperties of the object and the second set of display modifiers but notthe first set of display modifiers. By way of another example, the newvalue may include the display properties of the object and the secondset of display modifiers and the first set of display modifiers. By wayof yet another example, the new value may include the display propertiesand one or more of the first set of display modifiers and/or one or moreof the second set of display modifiers. By way of still another example,the new value may include a constant or one or more of the first set ofdisplay modifiers and/or the second set of display modifiers but not thedisplay properties of the object.

In the fifth operation 250, the display generator 105 may calculate anew display layout for the second node relative to the first node basedon the new value of the layout bound.

In the sixth operation 260, the display generator 105 may render atleast the first node and the second node to generate the display on thedisplay device 102 according to the calculated new display layout.

By way of example, with reference to FIGS. 3A-3C, performance of anexample implementation of method 200 is illustrated. In FIG. 3A, atwo-dimensional graphical user interface 300 is illustrated asdisplayed, having been rendered from a scene graph utilizing the scenegraph application programming interface (API) of JavaFX™. The userinterface 300 includes a number of picture thumbnails 301, eachcorresponding to a node in the scene graph. The user interface 300,including the picture thumbnails 301, was generated when the nodes ofthe scene graph were rendered.

In the scene graph API of JavaFX™, nodes in a scene graph include alocal bound variable and a transforms [ ] variable. The local boundvariable specifies the area (such as the height, width, and/or position)of the object corresponding to the node. The transforms[ ] variablespecifies mathematical transformations that will be applied to modifythe object when the node is rendered, such as mathematicaltransformations to scale, skew, translate, reflect, shear, and/or applya visual effect or animation to the object. The nodes in the scene graphin the JavaFX™ scene graph API also include other variables (such asscaleX, scaleY, rotate, etc.), which specify mathematicaltransformations that will be applied to modify the object when the nodeis rendered. The transforms [ ] variable and the other variables may beutilized to apply the same mathematical transformations to modify theobject when the node is rendered. Thus, either the transforms [ ]variable or the other variables may be utilized in a particular instanceto apply a mathematical transformation to modify the object when thenode is rendered. However, the JavaFX™ scene graph API applies themathematical transformations specified in the other variables in apre-determined order, but applies the mathematical transformationsspecified in the transformations [ ] variable in whatever order they arespecified in the node. Thus, mathematical transformations may bespecified using the transformations [ ] variable, rather than the othervariables, when it is desired to apply mathematical transformations in aparticular order contrary to the pre-determined order of the othervariables.

In the present example, each of the nodes in the scene graph includes alayoutBounds variable. The layoutBounds variable is assigned a defaultvalue of the local bound variable and the transforms [ ] variable, butnot the other variables. Also in the present example, the scene graphincludes a specification to lay out each of the nodes at least a certaindistance away from the surrounding nodes. Thus, the layout is calculatedfor each of the nodes based on the layoutBounds of the surrounding nodesand the user interface 300 is rendered based on the calculated layout,resulting in the space between the picture thumbnails 301 in the userinterface 300.

In this example, when a user selects one of the picture thumbnails 301A,a mathematical scale transformation is set using the scaleX and scale Yvariables of the node in the scene graph corresponding to thatparticular picture thumbnail. As a result, the particular picturethumbnail 301B is scaled larger to illustrate that the particularpicture thumbnail 301A has been selected by the user. FIG. 3Billustrates the user interface 300 where a user has selected the centerpicture thumbnail 301A. As a result, the mathematical scaletransformation was set using the scaleX and scaleY variables of the nodecorresponding to the center picture thumbnail 301A and the centerpicture thumbnail 301B is rendered as scaled larger. However, the layoutof the other picture thumbnails 301 have not changed. This is becausethe layoutBounds variable does not include mathematical transformationsset in the other variables in the nodes, including the scaleX and scaleYvariables. Although the present example illustrates the scaled largercenter picture thumbnail 301B as blocking view of the other picturethumbnails, in some implementations other mathematical transformationsmay be applied when the user selected the center picture thumbnail 301to reduce the opacity of the center picture thumbnail 301 such that theother picture thumbnails 301 are visible behind the scaled larger centerpicture thumbnail 301 without departing from the scope of the presentdisclosure.

By way of contrast, when a user utilizes a zoom function of the userinterface 300, a mathematical transformation is set in each of the nodesusing the transforms [ ] variable of each node. As a result, each of thepicture thumbnails 301 is scaled larger to illustrate that the user haszoomed in. FIG. 3C illustrates the user interface 300 where a user haszoomed in. As a result, the mathematical scale transformation was setusing the transforms [ ] variable of each node and each of the picturethumbnails 301 is rendered as scaled larger. Further, the layout of thepicture thumbnails 301 have changed. This is because the layoutBoundsvariable for each node includes mathematical transformations set in thetransforms [ ] variable of each node.

However, the layoutBounds for each node in the scene graph in thisexample may be assigned to a value other than the local bound variableand the transforms [ ] variable but not the other variables. ThelayoutBounds may be set to a constant, in which case the layout of thepicture thumbnails 301 would not change regardless what mathematicaltransformations were set in each of the nodes, whether utilizing thetransforms [ ] variable or the other variables. Alternatively, thelayoutBounds may be set to the local bound variable and the othervariables but not the transforms [ ] variable. In that case, if a userselected a particular picture thumbnail 301 the layout of the otherpicture thumbnails 301 would change (similar to FIG. 3C) and if the userzoomed in the layout of the other picture thumbnails 301 would notchange (similar to FIG. 3B).

It is understood that the above example is merely exemplary. The displaymay be a display other than a two-dimensional graphical user interface,or the user interface may include objects other than picture thumbnailswithout departing from the scope of the present disclosure.Additionally, though the above example refers to specific technologiessuch as JavaFX™, it is understood that other technologies could beutilized without departing from the scope of the present disclosure.

The present disclosure may provide apparatus, systems and methods forlayout of scene graphs for generating displays using node boundingareas. These various features enable layout of nodes in scene graphswith respect to each other that is flexible and allows visual effectsand animations to either be factored into that layout or not.

The description above includes example systems, methods, techniques,instruction sequences, and/or computer program products that embodytechniques of the present disclosure. However, it is understood that thedescribed disclosure may be practiced without these specific details.

In the present disclosure, the methods disclosed may be implemented assets of instructions or software readable by a device. Further, it isunderstood that the specific order or hierarchy of steps in the methodsdisclosed are examples of exemplary approaches. Based upon designpreferences, it is understood that the specific order or hierarchy ofsteps in the method can be rearranged while remaining within thedisclosed subject matter. The accompanying method claims presentelements of the various steps in a sample order, and are not necessarilymeant to be limited to the specific order or hierarchy presented.

The described disclosure may be provided as a computer program product,or software, that may include a machine-readable medium having storedthereon instructions, which may be used to program a computer system (orother electronic devices) to perform a process according to the presentdisclosure. A machine-readable medium includes any mechanism for storinginformation in a form (e.g., software, processing application) readablyby a machine (e.g., a computer). The machine-readable medium mayinclude, but is not limited to, magnetic storage medium (e.g., floppydiskette), optical storage medium (e.g., CD-ROM); magneto-opticalstorage medium, read only memory (ROM); random access memory (RAM);erasable programmable memory (e.g., EPROM and EEPROM); flash memory; orother types of medium suitable for storing electronic instructions.

It is believed that the present disclosure and many of its attendantadvantages will be understood by the foregoing description, and it willbe apparent that various changes may be made in the form, constructionand arrangement of the components without departing from the disclosedsubject matter or without sacrificing all of its material advantages.The form described is merely explanatory, and it is the intention of thefollowing claims to encompass and include such changes.

While the present disclosure has been described with reference tovarious embodiments, it will be understood that these embodiments areillustrative and that the scope of the disclosure is not limited tothem. Many variations, modifications, additions, and improvements arepossible. More generally, embodiments in accordance with the presentdisclosure have been described in the context or particular embodiments.Functionality may be separated or combined in blocks differently invarious embodiments of the disclosure or described with differentterminology. These and other variations, modifications, additions, andimprovements may fall within the scope of the disclosure as defined inthe claims that follow.

The invention claimed is:
 1. A computer-implemented method, comprising:assigning, using at least one processing device, a first value to alayout bound variable of a layout bound, the layout bound defining abounding volume for an object, the object corresponding to a first nodein a plurality of nodes in a scene graph, wherein the first node of thescene graph includes: at least one display property defining at least anarea of the object of the first node; and at least one set of transformsfor modifying the at least one display property, the at least one set oftransforms comprising a first set of transforms and a second set oftransforms, and wherein the first value assigned to the layout boundvariable includes the area of the object and the first set oftransforms; using the at least one processing device, calculating, basedon the first value assigned to the layout bound variable, a firstdisplay layout for at least a second node in the plurality of nodes, thefirst display layout defining a first position of the second noderelative to the first node when rendered; first rendering, using the atleast one processing device, the first node and the second node by:applying the first set of transforms of the first value to the area ofthe object to thereby modify the object; and generating a first displayon a display device according to the calculated first display layout;assigning, using the at least one processing device, a second value tothe layout bound variable, wherein the second value comprises the secondset of transforms and is thereby different than the first value;calculating, using the at least one processing device, a second displaylayout for the second node relative to the first node based on thesecond value of the layout bound variable, the second display layoutdefining a second position of the second node relative to the first nodewhen rendered; and second rendering, using the at least one processingdevice, the first node and the second node by: applying the second setof transforms of the second value to the at least one display property,thereby modifying the object; and generating a second display on thedisplay device according to the second display layout.
 2. Thecomputer-implemented method of claim 1, wherein altering a transform inthe first set of transforms changes the first display layout.
 3. Thecomputer-implemented method of claim 1, wherein altering a transform inthe second set of transforms does not change the first display layout.4. The computer-implemented method of claim 1, wherein the displaydevice comprises a user interface.
 5. A non-transitory computer readablemedium encoded with instructions executable by a processor, theinstructions comprising: assigning a first value to a layout boundvariable of a layout bound, the layout bound defining a bounding volumefor an object, the object corresponding to a first node in a pluralityof nodes in a scene graph, wherein the first node of the scene graphincludes: at least one display property defining at least an area of theobject of the first node; and at least one set of transforms formodifying the at least one display property, the at least one set oftransforms comprising a first set of transforms and a second set oftransforms, and wherein the first value assigned to the layout boundvariable includes the area of the object and the first set oftransforms; calculating, based on the first value assigned to the layoutbound variable, a first display layout for at least a second node in theplurality of nodes, the first display layout defining a first positionof the second node relative to the first node when rendered; firstrendering, using the at least one processing device, the first node andthe second node by: applying the first set of transforms of the firstvalue to the area of the object to thereby modify the object duringrendering; and generating a first display on a display device accordingto the calculated first display layout; assigning a second value to thelayout bound variable, wherein the second value comprises the second setof transforms and is thereby different than the first value;calculating, using the at least one processing device, a second displaylayout for the second node relative to the first node based on thesecond value of the layout bound variable, the second display layoutdefining a second position of the second node relative to the first nodewhen rendered; and second rendering, using the at least one processingdevice, the first node and the second node by: applying the second setof transforms of the second value to the object to thereby modify theobject rendering; and generating a second display on the display deviceaccording to the second display layout.
 6. The non-transitory computerreadable medium of claim 5, wherein altering a transform in the firstset of transforms changes the first display layout.
 7. Thenon-transitory computer-implemented method of claim 5, wherein alteringa transform in the second set of transforms does not change the firstdisplay layout.
 8. The non-transitory computer readable medium of claim5, wherein the display device comprises a user interface.